Using `jq`

jq is a filter that generates output from input (which is usually a JSON).

Suppose we use the following json as an exmaple, through cat summary.json | jq <filters>

{
  "best_validation_loss": 0.8036915083726247,
  "test": {
    "num_samples": 60,
    "num_parsed": 60,
    "parse_rate": 1.0,
    "arousal": {
      "mae": 0.4794871794871795,
      "rmse": 0.7612405529707279,
      "accuracy": 0.5705128205128205,
      "macro_f1": 0.2384203989789512,
      "pearson": 0.4061388611777579,
      "ccc": 0.3555133079847911,
      "exact_match": false,
      "per_sample_exact_match_rate": 0.11666666666666667
    },
    "valence": {
      "mae": 0.05384615384615385,
      "rmse": 0.23204774044612855,
      "accuracy": 0.9461538461538461,
      "macro_f1": 0.19446640316205535,
      "pearson": null,
      "ccc": 0.0,
      "exact_match": false,
      "per_sample_exact_match_rate": 0.6666666666666666
    },
    "combined": {
      "mae": 0.26666666666666666,
      "rmse": 0.4966441467084282,
      "accuracy": 0.7583333333333333,
      "macro_f1": 0.21644340107050328,
      "pearson": 0.4061388611777579,
      "ccc": 0.17775665399239554,
      "per_sample_exact_match_rate": 0.39166666666666666
    },
    "rouge_l": 0.2915678760020875,
    "bertscore_precision": 0.8994393934806187,
    "bertscore_recall": 0.8893563201030096,
    "bertscore_f1": 0.8943329095840454,
    "undefined_correlations": 167,
    "loss": 0.8233803997437159
  }
}

Identity

It’s just a simple dot . that outputs the input.

$ cat summary.json | jq '.'
# same as the above

Object Field Index

We can access sub-object through field names, e.g., best_validation_loss, test. This can also be nested.

We can use dot notation .test, or bracket notation ["test"]. If the field name contains special characters, we must use bracket style indexing, ["<special>"]

$ cat summary.json | jq '.test'
# {
#   "mae": 0.4794871794871795,
#   "rmse": 0.7612405529707279,
#   "accuracy": 0.5705128205128205,
#   "macro_f1": 0.2384203989789512,
#   "pearson": 0.4061388611777579,
#   "ccc": 0.3555133079847911,
#   "exact_match": false,
#   "per_sample_exact_match_rate": 0.11666666666666667
# }

Array Indexing

We can index an array through bracket notation.

Using jq

Using `jq`